1. The Field of the Invention
The present invention relates to stress testing of network servers, wherein a plurality of concurrent network clients are simulated. More particularly, the present invention relates to stress testing of a network server in which multiple clients are simulated by executing one executable software thread on each of one or more processors, whereby I/O requests are initiated to the server on behalf of all clients. The stress testing of the present invention further includes executing another executable software thread on each of the one or more processors, whereby I/O response data from the server is received on behalf of all clients.
2. The Prior State of the Art
In recent years, use of servers has become common in both local area and wide area networks. Such networks allow multiple computer users to access the processing capabilities of the network server. In local area networks, network users can share information and gain access to software applications stored and executed in the server. Local area networks are also useful for providing rapid and convenient communication between users. Relatively inexpensive terminals can benefit from the typically more powerful computing capabilities of the server.
Wide area networks, such as the Internet, have become popular due to the ease with which network users at remote locations are able to retrieve and share information. For example, World Wide Web servers allow remote users to download text, graphics, multimedia materials, or to take advantage of network applications stored and executed by the server. Electronic mail servers present users with a quick and convenient means of telecommunication. Chat servers have also become increasingly popular in large measure for their ability to provide real-time communication between remote users. Users of a networked chat service can select from a great number of topics, join a chat service channel, and contribute to an ongoing discussion on the topic. Chat services can have substantial entertainment, informational, and educational value. It can be expected that in coming years, local area and wide area networks will become increasingly important as the related technology becomes more widespread and accepted.
One of the significant advantages of network servers is that they provide processing services to multiple concurrent clients. It is important for a network provider to ensure that a server is powerful enough to service a large number of simultaneous clients at peak times. Many network users, particularly of the Internet, have experienced delayed response times during hours of high usage. When a network provider fails to maintain a server with sufficient speed and multiple client capability, the inconvenienced users are often frustrated and can lose interest in the services of the network provider. However, it is often difficult to predict beforehand how a server will respond to high volumes of file uploads or downloads or other input/output (I/O) requests and client actions. Often, a network provider does not discover that server resources are inadequate until after a large number of clients are inconvenienced.
It is particularly important to ensure that sufficient server capabilities are on hand when a network provider stages a well publicized chat session. For example, it has become increasingly common for chat server providers to advertise for and provide sessions in which users may communicate with celebrities. In such cases, it is common to receive requests for access from remote users in volumes that are much greater than those of typical chat sessions. Network providers who conduct such large-volume chat sessions naturally want to have adequate server capabilities, but may find it difficult to know if their resources are sufficient. When hundreds or thousands of simultaneous users are expected to generate I/O requests to a server installation, it is impractical to conduct tests using actual clients, and it may be impossible to rely on past experience.
In order to predict whether a chat server, or any other network server, is powerful enough to handle peak volume of a specified magnitude, there have been developed test methods of simulating multiple concurrent network clients to the server. Seen in FIGS. 1 and 2 is one such method that is known in the art. FIG. 1 illustrates the arrangement and relationship of elements within a client simulator computer and the network, while FIG. 2 depicts the steps of the method in flow chart form.
Referring now to FIG. 1, at least one client simulator computer is connected to server 60 in order to provide simulated I/O requests. Executable software threads are initiated and dedicated to each of a plurality of simulated clients in a one-to-one relationship. Accordingly, if a test is designed to simulate 1,000 concurrent clients, there will be 1,000 dedicated client threads 62. Each dedicated client thread 62 is associated with a client profile 64 for one of the simulated clients. Dedicated client thread 62 initiates and sends I/O requests on behalf of its simulated client to an associated socket 68. The I/O requests are forwarded through network communication infrastructure 70 to server 60. A switching module 66, such as a context switching routine or the like, is used to assign one of the dedicated client threads 62 to the processor of the client simulator computer. If the client simulator computer has only one processor, only one of dedicated client threads 62 may run at any one time.
Each client profile 64 typically assigns to the simulated client a set of simulated characteristics, such as the frequency of making I/O requests, the type of I/O requests to be made, and the states that are possible. For example, a first simulated client may be designated as a normal chat client who makes I/O requests every 30 seconds on average. The first simulated client may be allowed to join a chat channel from the logged-on state and to make a series of I/O request, 95% of which may be talk messages to the server, with 5% being requests to part or exit the channel. A second simulated client may be designated as one who joins the channel and does little more than read posted messages while contributing none of its own. Another simulated client may be designated as a host, while still another may be assigned to be the systems operator (sysop). In any event, client profiles 64 are designed to mimic the actions that actual network clients are likely to make.
Each dedicated client thread 62 repeatedly reads and updates the information stored in the associated client profile 64. In operation, a dedicated client thread 62 executes step 72 of FIG. 2 by reading and evaluating the state of its simulated client from the associated client profile 64. For example, it may be determined that the simulated client is in a logged-on state. Next, in step 74, dedicated client thread 62 selects an I/O request for its simulated client from among the permissible requests for a client in the logged-on state. For example, a request to join a chat channel may be selected. In step 78, the I/O request is initiated and sent to an associated socket 68 and in turn sent through the network communication infrastructure 70 to the server 60.
After sending an I/O request, the dedicated client thread 62 waits, in step 80, for a response from server 60. When the I/O request is completed, I/O response data thereby generated is sent from server 60 back to the appropriate socket 68 through network communication infrastructure 70. The associated dedicated client thread 62 detects the response to the I/O request and retrieves information from its socket 68 in step 82. Dedicated client thread 62 then determines, according to step 84, whether the response indicates that the state of the associated simulated client should be changed, for example, from logged-in to in-channel. According to step 86, if no change is to be made, the routine reverts to step 72. If the simulated client should be moved to a new state, the appropriate changes are made in client profile 64, as shown in step 88, after which dedicated client thread 62 is ready to begin the cycle once again. At any time during execution of the foregoing steps, dedicated client thread 62 is generally subject to losing and regaining access to the processor according to a switching protocol executed by switching module 66.
A suitable stress test should satisfy at least four criteria. First, the simulated clients must perform a wide variety of actions that closely approximate the behavior of actual clients. Second, the number of simulated clients must be as large as desired. Third, the method used for simulating clients must be fast enough to handle the response from the server so as not to create an artificial bottleneck. Finally, the method is preferably generic so that it can be customized to simulate a wide variety of clients and client behavior.
The foregoing prior art method of assigning each simulated client a dedicated thread becomes inefficient once the number of simulated clients becomes sufficiently large. Thus, the known methods do not satisfactorily meet the second of the criteria listed above in that the number of clients that may be simulated is limited. When there are many clients, context switching between threads consumes a large portion of the resources of the simulating machine. This can cause the results of the stress test to be inadequate, since the simulation may not be able to generate I/O requests as quickly as would be required. It has been found that an appreciable amount of computer resources may be consumed in context switching between as few as 40 dedicated client threads.
Further, conventional operating systems on which the prior art stress testing systems are conducted typically have structural limits to the number of executable threads that may be supported. For instance, Microsoft Windows NT scales to about 2,000 threads. Accordingly, stress tests running on a Windows NT system with only one processor have been limited to about 2,000 simulated clients. This is unsatisfactory for testing servers that may be called on to serve many times that number of actual clients.
It would, therefore, be desirable to have a stress testing method to simulate a number of clients that is larger than the number of threads supported by conventional operating systems. In particular, it would be advantageous to provide a stress testing method to simulate a number of clients that is not limited by the operating system or the context switching capabilities of the machine on which it runs. It would be advantageous to provide a method for initiating I/O requests for a large number of simulated clients wherein the amount of computer resources dedicated to switching between executable threads is minimized. It would also be desirable to have such a testing method that is flexible such that a wide variety of servers could be tested.